Field of the Invention
The present invention relates to information handling systems. More specifically, embodiments of the invention relate to dynamically optimizing data access patterns using predictive crowdsourcing.
Description of the Related Art
As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
It is known to couple a plurality of information handling systems to provide a business intelligence architecture. In known business intelligence architectures, querying operations which retrieve data from disk storage is often the slowest part of data processing the business intelligence querying operations. Additionally, the more data required by components or users of the business intelligence architecture, the more the retrieval of that data slows down the analytics process. One method for addressing this issue is to pre-process data in some way (e.g., via cubes, query sets, aggregate tables, etc.) so information handling system using the data can retrieve a smaller number of records. However, pre-processing can often require accurate predictions of what data should be selected and how the data should be arranged for analysis. If and/or when more or different data is needed, a new prediction regarding pre-processing the data is needed.
Certain in-memory databases approach the querying issue by loading an entire dataset into faster access memory such as local non-volatile memory (e.g., random access memory (RAM)). Such a process can remove or reduce the need to access the disk to run queries, thus gaining an immediate and substantial performance advantage (because scanning data in the local non-volatile memory is orders of magnitude faster than reading the same data from a disk).
Some implications to using in-memory technology include the amount of data you can query with in-memory technology is limited by the amount of free non-volatile memory available, and there is almost always much less available non-volatile memory than available disk space. It is theoretically possible to add additional non-volatile memory, however the hardware for interacting with the database then becomes much more expensive. Additionally, the amount of required non-volatile memory is not only affected by the amount of data, but also by the number of people querying the data. For example, having five to ten users accessing the same in-memory business intelligence application could easily double the amount of non-volatile memory required for intermediate calculations that need to be performed to generate the query results. Other implications include having to re-load from disk to the non-volatile memory every time the information handling system reboots and not being able to use the information handling system for anything other than the particular data model as the non-volatile memory is substantially consumed with storing the data to execute a particular data model.
Accordingly, it would be desirable to provide alternate methods to circumvent some of these inherent limitations of known business intelligence architectures.